Papers with low-bias architectures
How Can We Accelerate Progress Towards Human-like Linguistic Generalization? (2020.acl-main)
Copied to clipboard
| Challenge: | a new evaluation paradigm, Pretraining-Agnostic Identically Distributed evaluation, is needed . authors argue that it rewards models that can be trained on massive amounts of data, several orders of magnitude more than a human can expect to be exposed to. |
| Approach: | a position paper describes and critiques the Pretraining-Agnostic Identically Distributed evaluation paradigm . paradigm favors simple, low-bias architectures that can be scaled to process vast amounts of data . authors advocate for supplementing or replacing PAID with paradigms that reward architectures . |
| Outcome: | a new evaluation paradigm favors simple, low-bias architectures that can be scaled to process vast amounts of data. a san francisco-based study finds that the paradigm rewards architectures which generalize as quickly and robustly as humans. |